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Abstract.  We  have  demonstrated  that  the  Maximum  Entropy  (ME)  principle  in  the  context  of 
Bayesian  probability  theory  can  be  used  to  derive  the  probability  distributions  of  those  processes 
characterized  by  its  scaling  properties  including  multiscaling  moments  and  geometric  mean.  We 
started  from  a  proof-of-concept  case  of  a  power-law  probability  distribution,  followed  by  the 
general  case  of  multifractality  aided  by  the  wavelet  representation  of  the  cascade  model.  The 
ME  formalism  leads  to  the  probability  distribution  of  the  multiscaling  parameter  and  those  of 
incremental  multifractal  processes  at  different  scales.  Compared  to  other  methods,  the  ME  method 
significantly  reduces  computational  cost  by  leaving  out  unimportant  details.  The  ME  distributions 
have  been  evaluated  against  the  empirical  histograms  derived  from  the  drainage  area  of  river 
network,  soil  moisture  and  topography.  This  analysis  supports  the  assertion  that  the  ME  principle 
is  a  universal  and  unified  framework  for  modeling  processes  governed  by  scale-invariant  laws.  The 
ME  theory  opens  new  possibilities  of  extracting  information  of  multifractal  processes  beyond  the 
scales  of  observation. 

Keywords:  Bayesian  Statistics,  Maximum  Entropy,  Scale-Invariant  Laws,  Multifractality,  Environ¬ 
mental  Sciences. 

PACS:  02.50.-r,  05.45.Df 


INTRODUCTION 

This  paper  reports  the  investigation  of  two  recent  studies  [1,  2],  motivated  by  the  need 
to  show  the  practical  application  of  the  Maximum  Entropy  (ME)  principle  [3,  4,  5]  in 
the  description  of  statistics  of  multiscaling  processes.  These  processes  are  considered: 
1.  Drainage  Area  of  River  Network  (DARN),  2.  Incremental  surface  soil  moisture  and 
topography  fields  viewed  as  a  Self-similar  Processes  (SP),  3.  Incremental  topography 
as  a  Multifractal  Process  (MP).  Our  goal  is  to  establish  the  equivalence  of  both  the 
empirical  and  ME  distributions,  yet  ensuring  computational  efficiency.  Instead  of  com¬ 
puting  empirical  histograms  from  large  amount  of  data,  only  some  details  are  relevant 
to  capture  the  macroscopic  behaviour  by  the  Bayesian  analysis.  Compared  to  other  mul¬ 
tifractal  models,  the  ME  predicts  the  probability  distribution  of  multiscaling  processes 
at  different  scales  using  fewer  parameters.  The  key  to  make  successful  predictions  lies 
in  the  correct  identification  of  essential  physical  constraints.  Based  on  previous  stud¬ 
ies  [6,  7,  8,  9,  10],  we  assume  that  the  geometric  structures  were  an  attractive  can¬ 
didate  in  determining  statistics.  It  is  well  known  that  variables  such  as  drainage  area, 
discharge  and  energy  release,  soil  moisture,  increments  of  topography,  etc.,  are  self¬ 
similar  characterized  by  the  multiscaling  moments  or  power-law  distributions.  For  the 
case  of  increments  of  topography,  the  multifractal  condition  is  also  an  important  scal¬ 
ing  property  described  by  means  of  the  wavelet  representation  of  the  cascade  model 
[11,  12,  13,  14,  15,  16,  9,  17,  18,  19].  It  should  be  noted  that,  even  though  the  physi- 
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cal  principles  underlying  various  multiscaling  processes  might  be  different,  the  statistics 
are  found  to  be  dominated  by  some  geometric  properties.  The  similar  patterns  that  these 
processes  exhibit  at  different  resolutions  lead  the  “most  probable”  configuration.  Our 
findings  also  shed  light  on  the  information  beyond  the  limited  observational  scales. 


RESULTS  AND  DISCUSSION 

Here,  we  first  summarize  the  previous  results  of  the  ME  distributions  of:  1.  DARN,  2. 
SP,  and  3.  MP.  The  procedure  involved  is  explained  in  Materials  and  methods,  where 
the  following  constraints  have  been  used:  i)  geometric  means,  ii)  multiscaling  moments, 
and  iii)  multifractal  condition.  Further  details  can  be  found  in  [1,  2]. 

1.  The  ME  distribution  for  a  given  DARN,  a,  with  A\  <  a  <  A^,  A\  and  A2  as  the 
upper  and  lower  limit  of  a,  respectively,  subject  to  constraint  i),  Ag,  is  the  power- 
law  distribution: 


p(a) 


1-jU 


a1 


a 


-M 


a) 


where  for  the  case  of  A2  A\,  the  Lagrangian  multpilier  p  is  a  function  of  Ag, 


p  =  l+  or  M  =  1  -  ( log^f)  ’ 

depending  on  whether  p  >  1  or  p  <  1. 

2.  Given  the  incremental  process  \zi  —  z.o\  of  a  SP  field  z(x),  where  x  is  a  two- 
dimensional  location  vector,  and  z\  =  z(x  1)  and  zi  =  z(x 2),  subject  to  i)  and  ii), 
the  ME  joint  distribution  can  be  expressed  as 


1  I  M  1 

P(zi,Z2)  =  ^\zi~Z2\  ^expj-J^  Pq\zi-Z2\q>,  (3) 

where  pq  is  the  Lagrangian  multiplier  associated  with  the  c/th  moment  constraint. 

3.  Denote  the  incremental  process  of  z  at  a  location  x  over  a  two-dimensional  domain 
by  A rz  =  \z\  —  Z2 1  where  z\  =  z{rx\ )  and  z'2  =  z(rx 2)  are  a  re-scaled  z  by  scale  factor 
r  >  0.  The  ME  joint  distribution  of  a  MP,  A rz,  p-(z'|  .Zo),  satisfying  the  three  scaling 
properties  (i),  ii)  and  iii)),  is  given  by 


1  f  n  M  j 

pM,Zi)  =  5; (Akz)-* exp  -  £  KKlP  -  E  W»(A ^)"  (4) 

L  \  n= 1  m=  1  J 

where  the  Lagrangian  multipliers  X„  and  pm  are  again  related  to  the  given  con¬ 
straints,  and  A,./,./  is  a  multiscaling  parameter  (the  cascade  variable  [20])  indepen¬ 
dent  of  Arz  and  A,./z  for  any  pair  of  r  and  r'  with  ME  distribution  as 
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(5) 


PA(Ar/r')  =  ^7  exp  |-  £  XnAnr/,  |  • 

Two  data  products  were  used  to  evaluate  the  ME  distributions  of  a  SP,  Eq.  (3),  against 
empirical  histograms  over  fifteen  regions:  the  AMSR-E  soil  moisture  L2B  product  and 
the  USGS  National  Elevation  Dataset  (NED).  The  latter  was  also  used  to  evaluate  the 
predicted  power-law  exponent  p,  Eq.  (2),  in  the  Grand  Canyon  area  and  some  river 
networks  of  Puerto  Rico,  as  well  as  Eqs.  (4)-  (5)  in  the  characterization  of  a  MP 
for  seven  regions.  For  the  case  1,  it  was  found  that  predicted  and  empirical  results 
(not  shown)  were  in  close  agreement,  revealing  the  link  between  statistics  and  the 
macroscopic  properties  of  the  system.  Similarly,  the  ME  distributions  of  soil  moisture 
and  elevation  increments  (case  2)  were  compared  with  the  empirical  ones  for  different 
separation  distances  and  M  =  2,  resulting  in  an  almost  identical  agreement.  Only  results 
for  the  case  of  topography  are  included  in  this  paper  (see  Fig.  (1)).  The  geographical 
coordinates  of  the  reported  region  are  [-83.10,-83.04]  in  east  longitude  and  [35.25,35.3] 
in  north  latitude.  The  same  region  is  shown  for  the  test  of  case  3  predictions  for  TV  =  2 
and  M  =  2.  In  this  case,  the  wavelet  analysis  ([21,  22])  was  applied  in  order  to  study 
the  distributions  at  several  scales  of  observation  and  different  directions  (see  Fig.  (2)), 
where  in  the  wavelet  domain:  z  becomes  a.  l  A rz  and  A /z  turns  into  Aa.  /  ^  and 
Aa .  i ,  respectively,  and  q . ,  ?  corresponds  to  Ar/fj.  Figures  (3)-  (4)  compare  the  ME 

and  empirical  distributions  with  log-log  regression  coefficients  of  around  0.99  and  0.87, 
respectively.  Higher  correlation  coefficients  are  found  at  any  scale  for  bigger  domains. 
As  far  as  we  know,  the  results  are  independent  of  the  type  of  wavelet  bases  (Haar, 
Daubechies,  Symmlet,  Coiflet,  Battle-Lemarie)  when  sufficient  wavelet  coefficients  are 
included.  Only  second  order  moment  is  considered  in  this  study.  A  better  agreement 
between  empirical  and  theoretical  distributions  is  expected  for  higher  moments  (i.e., 
M  >  2,  N  >  2). 


CONCLUSIONS 

This  study  re-examines  some  previously-raised  issues  concerning  the  derivation  of  prob¬ 
ability  distribution  of  multiscaling  processes  using  the  ME  principle  in  the  context  of 
Bayesian  probability  theory  [1,2],  Typically  probability  distributions  are  obtained  com¬ 
puting  histograms  that  require  a  considerable  number  of  data  points.  The  case  studies 
of  drainage  area  of  a  river  network,  soil  moisture  and  topography  suggest  that  the  ME 
yields  robust  macroscopic  statistics  with  tractably  reduced  set  of  physical  constraints 
that  gives  the  same  result  as  empirical  histograms.  Identification  of  those  constraints  is 
often  difficult  since  the  behaviour  of  the  complex  Earth  system  is  not  fully  understood. 
Nonetheless,  physical  processes  following  scaling  laws  manifest  some  geometric  pat¬ 
terns  that  reveal  underlying  mechanisms.  Common  geometric  properties  associated  with 
scale-invariant  processes  include:  the  multifractal  condition,  the  multiscaling  moments, 
and  the  geometric  means.  ME  translates  these  physical  assumptions  into  macroscopic 
predictions.  Because  the  ME  discards  irrelevant  information,  it  also  improves  computa¬ 
tional  efficiency.  For  a  multifractal  system  such  as  topography,  in  addition  to  the  prob- 
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FIGURE  1.  From  left  to  right:  (a)  1  arc  second  NED  map  represented  in  longitude  A  and  latitude  6 
degrees  for  a  domain  of  128  x  128  points,  (b)  and  (c)  Empirical  (Pe)  and  the  MaxEnt  distributions  for 
M  =  2  (Pt)  according  to  Eq.  (3).  All  probabilities  are  plotted  versus  the  absolute  value  of  the  increments 
Az  =  \z(x\ )  —z(x 2)|  for  different  separation  distances  \x\  —  ^|. 
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FIGURE  2.  Wavelet  multiresolution  representation  of  the  NED  map  shown  in  Fig.l  (a)  for  two  levels 
of  decomposition  using  Haar  basis  ( a .  t  j).  From  top  to  bottom:  re-scaled  Fig.l  (a)  map  into  64  x  64  (a,  b 
and  c)  and  32  x  32  (d,  e  and  f)  points  domains  (i.e.,  j  =  1  and  2).  From  left  to  right:  horizontal  (a  and  d), 
vertical  (b  and  e)  and  diagonal  (c  and  f)  details  (i.e.,  /  =  1,2,  3). 


ability  distribution  of  the  incremental  process,  the  ME  predicts  the  probability  distri¬ 
bution  of  the  multiscaling  parameter  (the  cascade  variable)  using  the  wavelet  analysis 
of  the  cascade  model.  These  distributions  are  used,  for  instance,  in  the  study  of  fluvial 
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FIGURE  3.  Empirical  (Pe)  and  predicted  ME  distributions  for  N  =  2  (Er  or  Eq.  (5)  in  the  wavelet 
domain)  of  the  random  variable  rj  .^  (i.e.,  Ar/r/)  which  characterizes  the  scaling  properties  of  a  multi¬ 
fractal  process.  According  to  the  wavelet  decomposition,  a,  b  and  c  stand  for  /  =  1,2  and  3,  respectively, 
from  the  scale  of  observation  j  =  1  to  the  j  =  2  one  (downscale  cascade).  Results  correspond  to  order  3 
Battle-Lemarie  basis.  Correlation  coefficients  are  of  around  0.99. 


FIGURE  4.  Probability  distributions  of  increments  of  topography  Aa .  t  (i.e.,  A/z)  as  a  function  of 

j- U>2 

T]j  j  j  (i.e.,  Ar/r/)  for  a  given  separation  distance  |ii  —  *2|.  From  top  to  bottom:  empirical  (Pe  for  a,  b  and 
c)  and  predicted  ME  distributions  for  N  =  2  and  M  =  2  (Pt  or  Eq.  (4)  in  the  wavelet  domain  for  d,  e  and 
f),  both  at  j  =  2.  From  left  to  right:  /  =  1,2  and  3  details,  respectively.  Results  correspond  to  order  3 
Battle-Lemarie  basis.  Correlation  coefficients  are  of  around  0.87. 


geomorphology,  where  the  evolution  of  landscapes  clearly  plays  an  important  role  since 
it  is  strongly  influenced  by  the  presence  of  storms,  i.e.,  large  floods  [13].  The  statistics 
under  all  possible  variations  of  elevation  and  changes  of  resolution  can  be  now  described 
from  partial  information  consistent  with  the  identified  constraints.  These  findings  open 
the  possibility  of  gathering  information  across  the  spatial  scales  outside  of  those  of  ob¬ 
servations  of  many  research  fields.  The  existence  of  the  cascade  model  is  not  confined 
to  this  particular  context  and  ME  can  be  applied  to  other  multifractal  processes. 
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MATERIALS  AND  METHODS 


Data 

The  AMSR-E  L2B  soil  moisture  product  provides  estimates  of  soil  moisture  of  the 
top  1  cm  derived  from  brightness  temperature  at  10.7  GHz  with  resolution  of  38  km, 
re-gridded  into  a  global  cylindrical  25  km  Equal-Area  Scalable  Earth  Grid  (EASE- 
Grid).  The  NED  raster  elevation  data  of  one  arc-second  resolution  (30  m)  over  the 
continental  US  are  derived  from  multiple  satellites.  NAD83  and  NAV88  are  consistently 
used  as  horizontal  and  vertical  datums,  respectively,  and  all  the  data  are  re-gridded 
using  a  geographic  projection.  Soil  moisture  accuracy  (root-mean-square  difference) 
and  the  vertical  accuracy  (absolute  error)  of  the  NED  product  is  on  the  order  of  7- 
15  m.  More  details  can  be  found  at:  http://nsidc.org/data/amsre/,  and 
http://ned.usgs. gov/. 


Maximum  Entropy  Principle 

The  maximum  entropy  (ME)  principle  states  that  “out  of  all  the  possible  probabiliy 
distributions  which  agree  with  the  given  constraint  information,  select  the  one  that 
is  maximally  non-commital  with  regard  to  missing  information”  [5].  The  Shannon 
information  entropy  Sj  of  a  probability  density  function  p(x)  of  a  continuous  variable  x 
is  defined  as  [23,  4,  5], 


Si  =  - 


P(x) 

m{x)\ 


dx, 


(6) 


where  m(x)  is  the  ignorance  prior  [4]. 

Suppose  that  some  testable  information,  (i.e.,  set  of  available  experimental  results  or 
conserved  quantities)  can  be  expressed  in  terms  of  constraints  in  the  form, 


J fk(x)p(x)dx  =  Fk,  k  =  0,...,M.  (7) 

where  fk  is  an  arbitrary  function  of  x,  and  Fk  the  given  parameters. 

Maximizing  Sj  in  Eq.  (6)  subject  to  the  constraints  of  Eq.  (7)  leads  to  a  (normalized) 
probability  density  function, 


p(x)  = 


m(x) 


M 


eXP  )  L  Pkfk(x) 

k= 0 


where  /4  is  the  Lagrangian  multiplier  associated  with  the  constraint  Fk  through, 

<9logZ 


Fk  = 


dpk 


(8) 


(9) 
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and  Z  is  the  well-known  partition  function. 


f  M  'l 

m(x)  exp  —  Yj  Hkfk(x)  >  dx.  (10) 

l  k= o  J 

p(x)  in  Eq.  (8)  is  referred  to  as  the  "ME  distribution".  In  this  study,  we  assume  m(x)  =  1. 
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Z(pq,...,Pm)  =  J 
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